|
|
|
| Shifted Label Proportion Aware Semi-supervised Support Vector Machine |
| LI Yuanzhao, WANG Shaobo, LI Yufeng |
| State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210023 |
|
|
|
|
Abstract When the label proportion of unlabeled data is far away from that of labeled data, direct supervised support vector machine(SVM) with only labeled data outperforms semi-supervised SVM(S3VM) with unlabeled data. Thus, a shifted label proportion aware S3VM(fairS3VM) is proposed. Specifically, the label mean of unlabeled data is firstly estimated. Then multiple label means corresponding to multiple label proportions are integrated under the worst-case scenario. Experimental results show that the performance guarantee of S3VMs is effectively improved when the label proportion is shifted.
|
|
Received: 02 March 2016
|
|
|
| About author:: LI Yuanzhao, born in 1992, master student. His research interests include machine learning and data mining.WANG Shaobo, born in 1990, master student. His research interests include machine learning.LI Yufeng(Corresponding author), born in 1983, Ph.D., assistant professor. His research interests include machine learning and data mining. |
|
|
|
[1] CHAPELLE O, SCHLKOPF B, ZIEN A. Semi-supervised Lear-ning. Cambridge, USA: MIT Press, 2006.
[2] ZHOU Z H, LI M. Semi-supervised Learning by Disagreement. Knowledge and Information Systems, 2010, 24(3): 415-439.
[3] ZHU X J. Semi-supervised Learning Literature Survey [J/OL].[2016-02-28].http://pages.cs.wisc.edu/~jerryzhu/pub/ssl_survey.pdf.
[4] LIU W, WANG J, CHANG S F. Robust and Scalable Graph-Based Semisupervised Learning. Proceeding of the IEEE. 2012, 100(9): 2624-2638.
[5] 周志华.基于分歧的半监督学习.自动化学报, 2013, 39(11): 1871-1878.
(ZHOU Z H. Disagreement-Based Semi-supervised Learning. Acta Automatica Sinica, 2013, 39(11): 1871-1878.)
[6] JOACHIMS T. Transductive Inference for Text Classification Using Support Vector Machines // Proc of the 16th International Confe-rence on Machine Learning. San Francisco, USA: Morgan Kaufmann Publishers, 1999: 200-209.
[7] WANG L, CHAN K L, ZHANG Z H. Bootstrapping SVM Active Learning by Incorporating Unlabelled Images for Image Retrieval // Proc of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2003, I: 629-634.
[8] KASABOV N, PANG S N. Transductive Support Vector Machines and Applications in Bioinformatics for Promoter Recognition // Proc of the International Conference on Neural Networks and Signal Processing. Nanjing, China: IEEE, 2003, I: 1-6.
[9] GOUTTE C, DJEAN H, GAUSSIER E, et al. Combining Labelled and Unlabelled Data: A Case Study on Fisher Kernels and Transductive Inference for Biological Entity Recognition // Proc of the 6th Conference on Natural Language Learning. Stroudsburg, USA: Association for Computational Linguistics, 2002, 20: 1-7.
[10] BENNETT K P, DEMIRIZ A. Semi-supervised Support Vector Machines // KEARNS M J, SOLLA S A, COHN D A, eds. Advances in Neural Information Processing Systems 11. Cambridge, USA: MIT Press, 1999: 368-374.
[11] VAPNIK V N. Statistical Learning Theory. New York, USA: Wiely, 1998.
[12] VAPNIK V N. The Nature of Statistical Learning Theory. New York, USA: Springer-Verlag, 1995.
[13] VAPNIK V N, STERIN A M. On Structural Risk Minimization or Overall Risk in a Problem of Pattern Recognition. Automation and Remote Control, 1977, 10(3): 1495-1503.
[14] CHAPELLE O, SINDHWANI V, KEERTHI S S. Optimization Techniques for Semi-supervised Support Vector Machines. Journal of Machine Learning Research, 2008, 9: 203-233.
[15] CHAPELLE O, SINDHWANI V, KEERTHI S S. Branch and Bound for Semi-supervised Support Vector Machines [EB/OL].[2015-12-02]. http://www.keerthis.com/bb_nips_chapelle_06.pdf.
[16] LI Y F, KWOK J T, ZHOU Z H. Semi-supervised Learning Using Label Mean // Proc of the 26th Annual International Conference on Machine Learning. New York, USA: ACM, 2009: 633-640.
[17] COLLOBERT R, SINZ F, WESTON J, et al. Large Scale Tran-sductive SVMs. Journal of Machine Learning Research, 2006, 7: 1687-1712.
[18] BIE T D, CRISTIANINI N. Convex Methods for Transduction // THRUN S, SAUL L K, SCHLKOPF B, eds. Advances in Neural Information Processing Systems 16. Cambridge, USA: MIT Press, 2004: 73-80.
[19] XU L L, SCHUURMANS D. Unsupervised and Semi-supervised Multi-class Support Vector Machines // Proc of the 20th National Conference on Artificial Intelligence.Palo Alto, USA: AAAI Press, 2005, II: 904-910.
[20] CHAWLA N V, KARAKOULAS G. Learning from Labeled and Unlabeled Data: An Empirical Study across Techniques and Domains. Journal of Artificial Intelligence Research, 2011, 23: 331-366.
[21] ZHANG T, OLES F J. A Probability Analysis on the Value of Unlabeled Data for Classification Problems // Proc of the 17th International Confe-rence on Machine Learning. New York, USA: ACM, 2000: 1191-1198
[22] LI Y F, TSANG I W, KWOK J T, et al. Convex and Scalable Weakly Labeled SVMs. Journal of Machine Learning Research, 2013, 14: 2151-2188.
[23] NIGAM K, MCCALLUM A K, THRUN S, et al. Text Classification from Labeled and Unlabeled Documents Using EM. Machine Learning, 2000, 39: 103-134.
[24] BLUM A, CHAWLA S. Learning from Labeled and Unlabeled Data Using Graph Mincuts // Proc of the 18th International Confe-rence on Machine Learning. San Francisco, USA: Morgan Kau-fmann Publishers, 2001: 19-26.
[25] CHEN K, WANG S H. Semi-supervised Learning via Regularized Boosting Working on Multiple Semi-supervised Assumptions. IEEE Trans on Pattern Analysis and Machine Intelligence, 2011, 33(1): 129-143.
[26] LI Y F, ZHOU Z H. Towards Making Unlabeled Data Never Hurt. IEEE Trans on Pattern Analysis and Machine Intelligence, 2015, 37(1): 175-188.
[27] FAN R E, CHANG K W, HSIEH C J, et al. LIBLINEAR: A Library for Large Linear Classification. Journal of Machine Learning Research, 2008, 9: 1871-1874. |
|
|
|
2016, Vol. 29 
